I. Executive Summary
The mutually beneficial relationship between open standards and open source software has been instrumental in the update of geospatial technologies across a wide variety of industries. Whereas open source software has enabled many organizations to rapidly implement open standards, open standards have enabled open source software to support a wide variety of use cases through efficient integration and greater interoperability. An enduring challenge however, has been how and when to bring developers from the open standards and open source software communities together.
From the 25th to the 27th of April 2023, the Open Geospatial Consortium (OGC), the Apache Software Foundation (ASF), and the Open Source Geospatial Foundation (OSGeo) held their third joint Open Standards and Open Source Software Code Sprint. The code sprint served to accelerate the support of open geospatial standards within the developer community.
The code sprint brought together developers of Open Standards, Open Source Software and Proprietary Software into the same physical space. Such face-to-face collaboration had been made possible by the support of Ordnance Survey, the event’s Gold-level Sponsor, and the support of Camptocamp, the event host, as well as co-sponsors of the catering namely HEIG-VD (School of Engineering and Management), EPFL ,University of Lausanne , State of Neuchâtel , State of Vaud and Camptocamp.
Previous code sprints had been completely virtual events. Therefore this third Open Standards and Open Source Software Code Sprint held special significance in that it was the first hybrid event (supporting both face-to-face and remote participation) that had been jointly organized by the three organizations. This Engineering Report (ER) summarizes the main achievements of the code sprint.
The OGC is an international consortium of more than 500 businesses, government agencies, research organizations, and universities driven to make geospatial (location) information and services FAIR — Findable, Accessible, Interoperable, and Reusable. The consortium consists of Standards Working Groups (SWGs) that have responsibility for designing a candidate standard prior to approval as an OGC Standard and for making revisions to an existing OGC Standard. The sprint objectives for the SWGs were to:
Develop prototype implementations of OGC standards, including implementations of draft OGC Application Programming Interface (API) standards
Test the prototype implementations
Provide feedback to the Editor about what worked and what did not
Provide feedback about the specification document
The Open Source Geospatial Foundation (OSGeo) is a not-for-profit organization whose mission is to foster global adoption of open geospatial technology by being an inclusive software foundation devoted to an open philosophy and participatory community driven development. The foundation consists of projects that develop open source software products. The sprint objectives for OSGeo projects were:
TBA
TBA
The Apache Software Foundation (ASF) is an all-volunteer community comprising 815 individual Members and 8,500 Committers on six continents stewarding more than 227 million lines of code, and overseeing more than 350 Apache projects and their communities. The sprint objectives for ASF projects were:
TBA
TBA
The code sprint provided an environment for development and testing of prototype implementations of open standards, including implementations of draft and approved OGC Standards. Further, the code sprint provided a collaborative environment for developers to fix open issues in products, develop new features, improve documentation, improve interoperability with other libraries/products, and develop prototype implementations of OGC standards. Furthermore, the code sprint also enabled the participating developers to provide feedback to the editors of OGC standards.
The code sprint therefore met all of its objectives and achieved its goal of accelerating the support of open geospatial standards within the developer community.
The engineering report makes the following recommendations for future innovation work items:
TBA
TBA
The engineering report also makes the following recommendations for things that the Standards Working Groups should consider introducing support for:
TBA
TBA
II. Keywords
The following are keywords to be used by search engines and document catalogues.
OGC, OSGeo, ASF, Apache, hackathon, code sprint, standards, geospatial, API, open source
III. Security considerations
No security considerations have been made for this document.
IV. Submitters
All questions regarding this document should be directed to the editor or the contributors:
| Name | Organization | Role |
|---|---|---|
| Gobe Hobona | Open Geospatial Consortium | Editor |
| Joana Simoes | Open Geospatial Consortium | Editor |
| Angelos Tzotsos | Open Source Geospatial Foundation | Editor |
| Martin Desruisseaux | Geomatys | Editor |
| Tom Kralidis | Meteorological Service of Canada | Editor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
| TBA | TBA | Contributor |
V. Abstract
The subject of this Engineering Report (ER) is a code sprint that was held from the 25th to the 27th of April 2023 to advance support of open geospatial standards within the developer community, whilst also advancing the standards themselves. The code sprint was organized by the Open Geospatial Consortium (OGC), the Open Source Geospatial Foundation (OSGeo) and the Apache Software Foundation (ASF). The code sprint was sponsored by Ordnance Survey and hosted by Camptocamp.
2023 Open Standards and Open Source Software Code Sprint Summary Engineering Report
1. Scope
This Engineering Report (ER) summarizes the main achievements of the 3rd annual Open Standards and Open Source Software Code Sprint that was held from the 25th to the 27th of April 2023 in Bussigny, Switzerland. The ER presents the high level architecture, and describes each of the standards and software packages that were deployed in support of the code sprint. The ER also discusses the results and presents a set of conclusions, including recommendations.
2. Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.
Open API Initiative: OpenAPI Specification 3.0.3, https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md
Berners-Lee, T., Fielding, R., Masinter, L: IETF RFC 3896, Uniform Resource Identifier (URI): Generic Syntax, https://tools.ietf.org/rfc/rfc3896.txt
W3C: HTML5, W3C Recommendation, https://www.w3.org/TR/html5/
Schema.org: https://schema.org/docs/schemas.html
Clemens Portele, Panagiotis (Peter) A. Vretanos, Charles Heazel: OGC 17-069r4, OGC API — Features — Part 1: Core corrigendum. Open Geospatial Consortium (2022). https://docs.ogc.org/is/17-069r4/17-069r4.html.
Mark Burgoyne, David Blodgett, Charles Heazel, Chris Little: OGC 19-086r5, OGC API — Environmental Data Retrieval Standard. Open Geospatial Consortium (2022). https://docs.ogc.org/is/19-086r5/19-086r5.html.
Benjamin Pross, Panagiotis (Peter) A. Vretanos: OGC 18-062r2, OGC API — Processes — Part 1: Core. Open Geospatial Consortium (2021). https://docs.ogc.org/is/18-062r2/18-062r2.html.
Joan Masó, Jérôme Jacovella-St-Louis: OGC 20-057, OGC API — Tiles — Part 1: Core. Open Geospatial Consortium (2022). https://docs.ogc.org/is/20-057/20-057.html.
Charles Heazel: OGC API — Common — Part 1: Core, version 1.0.0, OGC 19-072, Open Geospatial Consortium, https://docs.ogc.org/is/19-072/19-072.html
3. Terms, definitions and abbreviated terms
This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.
This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.
For the purposes of this document, the following additional terms and definitions apply.
3.1. API
An Application Programming Interface (API) is a standard set of documented and supported functions and procedures that expose the capabilities or data of an operating system, application, or service to other applications (adapted from ISO/IEC TR 13066-2:2016).
3.2. coordinate reference system
A coordinate system that is related to the real world by a datum term name (source: ISO 19111)
3.3. OpenAPI Document
A document (or set of documents) that defines or describes an API. An OpenAPI definition uses and conforms to the OpenAPI Specification (https://www.openapis.org)
3.4. Web API
API using an architectural style that is founded on the technologies of the Web [source: OGC API — Features — Part 1: Core]
3.5. Abbreviated terms
API
Application Programming Interface
ASF
Apache Software Foundation
CITE
Compliance Interoperability & Testing Evaluation
CRS
Coordinate Reference System
CSW
Catalogue Services for the Web
EDR
Environmental Data Retrieval
GIS
Geographic Information System
MOU
Memorandum of Understanding
OGC
Open Geospatial Consortium
OSGeo
Open Source Geospatial Foundation
OWS
OGC Web Services
REST
Representational State Transfer
SDI
Spatial Data Infrastructure
TEAM
Test, Evaluation, And Measurement Engine
WCS
Web Coverage Service
WFS
Web Feature Service
WMS
Web Map Service
WMTS
Web Map Tile Service
WPS
Web Processing Service
4. High-Level Architecture
The focus of the sprint was on the support of implementations of open geospatial standards across various open source software projects. Implementations of approved and candidate OGC Standards were deployed in participants’ own infrastructure in order to build a solution with the architecture shown below in Figure 1.
Figure 1 — High Level Overview of the Sprint Architecture
As illustrated the sprint architecture was designed with the view of enabling client applications to connect to different servers that implement open geospatial standards such as the suite of OGC API standards. The architecture also included several different software libraries that support open geospatial standards and enable the extraction, transformation and loading of geospatial data. The rest of this section describes the software deployed and standards implemented during the code sprint.
4.1. Approved OGC Standards
4.1.1. OGC SensorThings API
The OGC SensorThings API provides an open and harmonized way to interconnect devices, data, and applications over the web and on the Internet of Things (IoT). At a high level the OGC SensorThings API provides two main parts, namely Part I — Sensing, and Part II — Tasking. The Sensing part provides a standard way to manage and retrieve observations and metadata from a variety of IoT sensor systems. The Tasking part provides a standard way for tasking IoT devices, such as sensors or actuators. The SensorThings API follows REST principles and uses JSON for encoding messages, as well as MQTT for publish/subscribe operations.
4.1.2. OGC API — Features
The OGC API — Features standard offers the capability to create, modify, and query spatial data on the Web and specifies requirements and recommendations for APIs that want to follow a standard way of sharing feature data. The specification is a multi-part standard. Part 1, labelled the Core, describes the mandatory capabilities that every implementing service has to support and is restricted to read-access to spatial data that is referenced to the World Geodetic System 1984 (WGS 84) Coordinate Reference System (CRS). Part 2 enables the use of different CRS, in addition to the WGS 84. Additional capabilities that address specific needs will be specified in additional parts. Envisaged future capabilities include, for example, support for creating and modifying data, more complex data models, and richer queries.
The OGC API — Features standard is part of the OGC API family of standards. OGC API standards define modular API building blocks to spatially enable Web APIs in a consistent way. The standards make use of the OpenAPI specification for defining the API building blocks.
4.1.3. OGC Styles and Symbology
The OGC Symbology Conceptual Model: Core Part Standard (OGC 18-067r3), also known as OGC SymCore, specifies the conceptual basis to define symbology rules for the portrayal of geographical data. It is modular and extensible (one core model, many extensions), also encoding agnostic (one symbology model, many encodings). It contains a minimal set of abstract classes representing explicit extension points of the model.
OGC Styles and Symbology is a vision for a standard that implements the OGC Symbology Conceptual Model: Core Part Standard (OGC 18-067r3). The Web Mapping Code Sprint was a key moment to consolidate the roadmap to achieve this vision. The plan towards such a candidate SymCore 2.0 Standard consists of the definition of a conceptual and logical model, Cascading Style Sheets (CSS) and JavaScript Object Notation (JSON) encodings, as well as a mapping to SLD/SE (eventually with existing well known vendor options). The requirements described will be supported by illustrated and encoded cartographic use cases.
4.1.4. OGC GeoXACML
GeoXACML defines a geo-specific extension to the XACML Policy Language, as defined by the OASIS standard “eXtensible Access Control Markup Language (XACML), that allows managing access to geographic information and services in an interoperable way across jurisdictions. Although XACML is meant to be used as a multi-purpose language, XACML does not have the capabilities to express geo-specific constraints on access rights. GeoXACML provides support for spatial data types and spatial authorization decision functions. Those data types and functions can be used to define additional spatial constraints for XACML based policies. GeoXACML defines extensions for the policy encoding schema only, and thus does not affect the XACML context schema.
Implementations of OGC API Standards are designed to work over HTTP(S). Securing the deployed APIs is not of concern to the OGC API Standards. However, the requirement to describe the API’s endpoint via OpenAPI introduces standard options to enable authentication. Knowing the user that accesses an API is an important aspect for many use cases. Also, user identification is often used to control access to the data, served by the API. But, no OGC API Standard provides guidance how to apply access control. This leads to an interoperability issue if access is denied, but there is no common (standardized) way to “tell the reason”. OGC’s GeoXACML 3.0 (currently still draft) Standard supports the interoperable exchange of access conditions as policies. GeoXACML 3.0 defines the geospatial extension to OASIS XACML 3.0 and thereby extends the ability to include Attribute Based Access Control also on geometries.
The challenge for the GeoXACML part of the code sprint was to develop a Policy Enforcement Point (PEP) that intercepts Web Feature Service (WFS) 2.0 requests and modifies the request according to an access decision returned by a GeoXACML 3.0 policy. The access control use case was the following: “WFS GetFeature requests shall exclude the NYC Central Park for feature type poly_landmarks”.
4.2. Candidate OGC Standards
4.2.1. OGC GeoPose
OGC GeoPose is an OGC Implementation Standard for exchanging the location and orientation of real or virtual geometric objects (Poses) within reference frames anchored to the earth’s surface (Geo) or within other astronomical coordinate systems. GeoPose specifies a single encoding in JSON format (IETF RFC 8259). The use cases addressed by GeoPose involve interactions between information systems or between an information system and a storage medium. The role of a GeoPose is to convey the orientation and position of a real or virtual object.
4.2.2. OGC API — Records
OGC API — Records provides discovery and access to metadata records about resources such as features, coverages, tiles / maps, models, assets, services or widgets. The draft specification enables the discovery of geospatial resources by standardizing the way collections of descriptive information about the resources (metadata) are exposed. The draft specification also enables the discovery and sharing of related resources that may be referenced from geospatial resources or their metadata by standardizing the way all kinds of records are exposed and managed. The draft OGC API – Records specification is part of the OGC API family of standards.
4.2.3. OGC GeoParquet
The Apache Parquet project provides an open source, columnar file format that supports both data storage and retrieval. The OGC GeoParquet specification defines how geospatial data should be stored in Parquet format, including the representation of geometries and the required additional metadata.
4.3. Other Specifications
4.3.1. AsyncAPI
AsyncAPI is a specification, by the Linux Foundation, for describing and documenting message-driven APIs in a machine-readable and protocol-agnostic format. The specification defines a set of files required to describe a message-driven API. The files describing the such an API in accordance with the AsyncAPI Specification may be encoded in JSON or YAML.
4.3.2. STAC API
The SpatioTemporal Asset Catalog (STAC) is a specification that offers a language for describing geospatial information, so it can be worked with, indexed, and discovered. The STAC API offers an interface that implements OGC API — Features. Although STAC has been developed outside of the OGC, in the long term it is envisaged that the STAC API specification will be developed into an OGC Community Standard that implements OGC API building blocks that are relevant for the STAC use cases.
4.4. ASF Apache Projects
4.4.1. Apache Arrow
Apache Arrow is a software framework for developing applications that can process columnar data in data analytics solutions. The software framework offers a column-oriented memory format that can represent flat and hierarchical data for analytic operations on modern CPU and GPU hardware. Apache Arrow supports the Apache Parquet Format, which is the format on which the OGC GeoParquet candidate standard is being built to extend.
4.4.2. Apache Baremaps
Apache Baremaps is an open source software toolkit and infrastructure components for creating, operating, and publishing maps online. The toolkit provides a data pipeline that enables developers to build maps from a variety of data sources. The toolkit enables cartographers to customize the content and the style of a map, as well as supporting services such as location search and IP to location.
4.4.3. Apache Sedona
Apache Sedona is a high-performance cluster computing system designed specifically for processing large-scale spatial data. The system leverages a number of open-source libraries to execute spatial operations such as Coordinate Reference System (CRS) transformation and file reading.
4.4.4. Apache SIS
Apache Spatial Information System (SIS) is a free and open source software library for developing geospatial applications. The library is an implementation of OGC GeoAPI 3.0.1 interfaces and can be used for desktop or server applications. Services provided by Apache SIS include metadata, coordinate transformations, filtering and grid coverages. The library is implemented using the Java programming language.
In this code sprint, Apache SIS was used for testing the Geospatial Integrity of Geoscience Software (GIGS) tests runner.
4.5. OSGeo Projects
4.5.1. OSGeo GeoNetwork
GeoNetwork is a catalog application for managing spatially referenced resources. It provides metadata editing and search functions as well as an interactive web map viewer.
4.5.2. OSGeo GeoServer
GeoServer is a Java-based software server that allows users to view and edit geospatial data. Using open standards by the OGC, GeoServer allows for great flexibility in map creation and data sharing.
4.5.3. OSGeo OpenLayers
OpenLayers is a library for developing browser-based mapping applications. It works with vector and raster data from a variety of sources and is able to reproject data for rendering. The library has broad support for OGC protocols and formats, including WMS, WMTS, Web Feature Service (WFS), Well Known Text (WKT), Well Known Binary (WKB), Geography Markup Language (GML), GeoTIFF/COG, and KML. In addition, it has support for community and non-OGC standards such as GeoJSON, XYZ tiles, TileJSON, and more. OpenLayers is written in JavaScript and is available under the BSD 2-Clause license.
The ol/source/OGCMapTile module provides a source for map tiles from an OGC API – Tiles implementation. The ol/source/OGCVectorTile module provides a source for vector tiles from an OGC API — Tiles implementation.
4.5.4. OSGeo OWSLib
OWSLib is a Python client for OGC Web Services and their related content models. The project is an OSGeo Community project and is released under a BSD 3-Clause License.
OWSLib supports numerous OGC standards, including increasing support for the OGC API suite of standards. The official documentation provides an overview of all supported standards.
4.5.5. OSGeo pycsw
pycsw is an OGC API — Records and OGC CSW server implementation written in Python. Started in 2010 (more formally announced in 2011), pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU), providing a standards-based metadata and catalogue component of spatial data infrastructures. pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X). pycsw is an official OSGeo Project as well as an OGC Reference Implementation.
pycsw supports numerous metadata content and API standards, including OGC API — Records — Part 1.0: Core and its associated specifications. The official documentation provides an overview of all supported standards.
4.5.6. OSGeo pygeoapi
pygeoapi is a Python server implementation of the OGC API suite of Standards. The project emerged as part of the next generation OGC API efforts in 2018 and provides the capability for organizations to deploy a RESTful OGC API endpoint using OpenAPI, GeoJSON, and HTML. pygeoapi is open source and released under an MIT license. pygeoapi is an official OSGeo Project as well as an OGC Reference Implementation.
pygeoapi supports numerous OGC API Standards. The official documentation provides an overview of all supported standards.
4.5.7. OSGeo QGIS
QGIS is a free and open-source cross-platform desktop GIS that supports viewing, editing, and analysis of geospatial data.
4.6. Community Open Source Projects
4.6.1. OSGeo ZOO-Project
ZOO-Project is a Web Processing Service (WPS) implementation written in C. It is an open source platform which implements the WPS 1.0.0 and WPS 2.0.0 OGC Standards.
4.6.2. TEAM Engine
The Test, Evaluation, And Measurement (TEAM) Engine is a testing facility that executes test suites developed using the TestNG framework or the OGC Compliance Test Language (CTL). It is typically used to verify specification compliance and is the official test harness of the OGC Compliance Testing Program (CITE).
4.6.3. CesiumJS
CesiumJS is an open source JavaScript library that supports visualization of 3D globes and maps on web browsers. The library supports user interaction and streaming in of 3D Tiles (an OGC Community Standard) and a variety of other formats.
4.6.4. OSGeo Leaflet
Leaflet is an open-source JavaScript library for mobile-friendly interactive maps. It works across all major desktop and mobile platforms, can be extended with a variety of plugins, and offers a well-documented API.
4.6.5. Camptocamp ogc-client
Camptocamp’s ogc-client product is a Typescript library that implements several OGC Standards. The library supports the Web Map Service (WMS), Web Feature Service (WFS), and OGC API — Features standards. The library also supports the OGC API — Records candidate standard.
4.6.6. HomeAssistant-SensorThings
Home Assistant is an award-winning free/libre/open-source home automation hub software written in Python. It can interface with over a thousand kinds of IoT devices and services, with a focus on domestic environments and data privacy.
HomeAssistant-SensorThings is a client for the OGC SensorThings standard (part 1) in the form of a Home Assistant plugin (or “integration” in Home Assistant jargon). It implements discovery of available sensors in an endpoint, and periodic updates of the values of those sensors.
4.6.7. Maplibre
MapLibre GL JS is an open-source library for publishing maps on websites or webview-based applications. The library uses GPU-accelerated rendering to offer client-side display of maps. The library supports client-side rendering of data, which makes it possible for an end-user to customise how a map is displayed.
4.6.8. OSGeo MDME
MDME (Model Driven Metadata Editor) is a web based application for populating YAML or JSON oriented metadata records. The application is built using Node/Vue.js. The application supports a variety of metadata models, for example MCF, OGC API — Records, and Data Package. The applications support extension of the internal metadata model.
4.6.9. OL-Cesium
OL-Cesium is a library that integrates OpenLayers and Cesium. The OL-Cesium library supports map contexts, raster data sources, vector data sources, map selection (selected items), and animated transitions between map and globe views.
4.6.10. OSGeo pygeometa
pygeometa provides a lightweight and Pythonic approach for users to easily create geospatial metadata in standards-based formats using simple configuration files (affectionately called metadata control files (MCF)). The software has minimal dependencies (install is less than 50 kB), and provides a flexible extension mechanism leveraging the Jinja2 templating system. Leveraging the simple but powerful YAML format, pygeometa can generate metadata in numerous standards. Users can also create their own custom metadata formats which can be plugged into pygeometa for custom metadata format output. pygeometa is open source and released under an MIT license.
For developers, pygeometa provides a Pythonic API that allows developers to tightly couple metadata generation within their systems and integrate nicely into metadata production pipelines.
The project supports various metadata formats out of the box including ISO 19115, the WMO Core Metadata Profile, and the WIGOS Metadata Standard. The project also supports the OGC API — Records core record model as well as STAC (Item).
4.6.11. Geomapfish
GeoMapFish is web application that enables developers to build extensible web-based GIS application. It supports functionality such as webmapping, editing, responsive templates, and customisation. The software supports a variety of OGC Standards, as well as the MapFish protocol for communication between client-side and server-side applications. The software is built using OpenLayers, AngularJS, ngeo, Papyrus, and other toolkits.
4.6.12. Smapshot
Smapshot is a crowdsourcing platform allowing volunteers to geolocalise in 3D historical images. More than 1’000 volunteers have geolocalised up to 250’000 images. An Open API is available to integrate the data: https://smapshot.heig-vd.ch/development_documentation and the API and georeferencer are open source.
4.6.13. go-ogc
go-ogc is a Go library for working with OGC API services. The library includes support for OGC API — Common, OGC API — Tiles, and OGC API — Features including support for the CQL2 JSON encoding. The library is available under the Apache 2.0 license.
4.6.14. GPQ
GPQ is a utility for working with GeoParquet files. The command line interface can be used to validate a GeoParquet file, describe its metadata, and can convert data to and from GeoJSON. The utility is also available as a WebAssembly binary for use in a browser. GPQ is written in Go and is available under the Apache 2.0 license.
5. Results
The code sprint included multiple software libraries, OWS implementations, OGC API implementations and different client applications. In addition to supporting OWS and OGC API standards, various ASF and OSGeo software products involved in the code sprint also supported a variety of OGC encoding standards. This section presents some of the results from the code sprint.
5.1. Approved OGC Standards
5.1.1. OGC SensorThings API
TBA
5.1.2. OGC API — Features
TBA
5.1.3. OGC Styles and Symbology
TBA
5.1.4. OGC GeoXACML
5.1.4.1. Purpose
Secure Dimensions implemented the GeoXACML 3.0 draft specification based on the Authzforce CE project which is an Open Source Community Edition project. The implementation is based on Java 11 and shall be made available as the Open Source reference implementation for GeoXACML 3.0. The implementation of a PEP during the Codesprint is important to test the GeoXACML 3.0 GeoPDP reference implementation by Secure Dimensions regarding request and response processing.
5.1.4.2. Proposed Solution
With respect to implementation, a Geoserver deployment including the default data will be used as the API that is to be protected. From the Geoserver architecture, different possibilities exist where to place the PEP. For the Codesprint, the solution to deploy a Tomcat Filter is favored. This ensures that the Geoserver installation can almost remain unchanged. The configuration of the “web.xml” must just be adopted to include the “WFS Filter for GeoXACML 3.0”. Also, the filter’s JAR and dependencies must be added to “geoserver.war”.
The implementation of the WFS Filter itself will use the Authzforce XACML-SDK for Java that needs to be extended to allow the inclusion of Geometry attributes. The actual access condition to exclude all features of the Central Park should be modelled in a GeoXACML 3.0 Policy. This is possible by constructing an Obligation on Permit that returns the OGC Filter condition that the PEP needs to apply to the intercepted request.
5.1.4.3. Achievement
During the Code Sprint, a demo was implemented that follows the XACML flow diagram.
Figure 2 — XACML Flow Diagram
A Geoserver 2.23 WAR-file deployment was used create the WFS 2.0 service endpoint.
The injection of the PEP was done by updating the “geoserver.war” file manually
zip -u geserver.war WEB-INF/web.xml
zip -u geoserver.war WEB-INF/lib/GeoXACMLFilter.jar
The web.xml file contains the deployment for the WFS Filter for GeoXACML 3.0.
<filter>
<filter-name>GeoXACMLFilter</filter-name>
<filter-class>de.securedimensions.geoxacml3.ows.GeoServerFilter</filter-class>
<init-param>
<param-name>pdpURL</param-name>
<param-value>https://ogc.demo.secure-dimensions.de/authzforce-ce</param-value>
</init-param>
<init-param>
<param-name>pdpDomain</param-name>
<param-value>A0bdIbmGEeWhFwcKrC9gSQ</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>Set Character Encoding</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Uncomment following filter to enable CORS
<filter-mapping>
<filter-name>cross-origin</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
-->
<filter-mapping>
<filter-name>GeoXACMLFilter</filter-name>
<url-pattern>/wfs</url-pattern>
</filter-mapping>
Geoserver web.xml initializing the WFS Filter for GeoXACML 3.0 on path /wfs
During the Codesprint one bug concerning the processing of XML attributes was fixed.
5.1.4.4. Demonstration
During the Code Sprint, the focus was to demonstrate the ability to rewrite WFS requests to reflect geospatial access control decisions. In order to achieve that, a simple GeoXACML 3.0 policy was crafted using ALFA for Visual Studio Code (ALFA plugin).
namespace ogc_codesprint {
import GeoXACML.*
import Attributes.*
obligation Filter = "urn:secd:wfs:filter"
policyset Manhattan = "root" {
target clause service == "WFS"
apply permitOverrides
policy POLY_LANDMARKS = "poly_landmarks" {
target clause feature_type == "tiger:poly_landmarks"
apply firstApplicable
rule CENTRAL_PARK {
target clause request == "GetFeature"
permit
condition(geometryOneAndOnly(bbox) >< "POLYGON((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464))":geometry)
on permit {
obligation Filter {
Attributes.Filter.operation = "Disjoint"
Attributes.Filter.geometry = "POLYGON((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464))":geometry
}
}
}
rule ALL_PERMIT {
permit
}
}
}
}GeoXACML 3.0 PolicySet described in ALFA
The GeoXACML 3.0 policy is structured in a simple way:
PolicySet (Manhattan) matches “service == WFS”
Policy (POLY_LANDMARKS) matches “typesNames == tiger:poly_landmarks”
Rule (CENTRAL_PARK) matches “request == GetFeature”
The Rule Condition contains is geospatial “BBOX Intersects Polygon(…)”
Any request that matches the condition results in the decision “Permit” with the Obligation “urn:secd:filter”. As specified in the XACML 3.0 specification, a PEP must enforce the decision including all obligations. The processing of this filter obligation provides the missing information to construct the WFS Filter (disjoint Central Park).
The result of this processing can be visualized with QGIS (WFS Layer):
Figure 3 — Left: Feature type “poly_landmarks” without PEP → Central Park feature(s) are included; Right: Feature type “poly_landmarks” with PEP → Central Park feature(s) are excluded!
The implementation of the Filter obtains the information from the HTTP request:
SubjectCategory subjectCat = new SubjectCategory();
ResourceCategory resourceCat = new ResourceCategory();
ActionCategory actionCategory = new ActionCategory();
EnvironmentCategory environmentCategory = new EnvironmentCategory();
AttributeValueType serviceType = new AttributeValueType(Arrays.asList(httpRequest.getParameter("SERVICE")), XACMLDatatypeId.STRING.value(), null);
Attribute service = new Attribute(Arrays.asList(serviceType),"urn:ogc:ows:service", "", false);
resourceCat.addAttribute(service);
Sample code for obtaining information from the HTTP request
Using the XACML-SDK for Java from Authzforce, the response from the PDP can be obtained in a few lines of code:
Request xacmlRequest = Utils.createXacmlRequest(Arrays.asList(subjectCat), Arrays.asList(resourceCat), Arrays.asList(actionCategory), Arrays.asList(environmentCategory));
ResponsesFactory xacmlResponse = pdp.getAuthZ(subjectCat, resourceCat, actionCategory, environmentCategory);
for (Response r : xacmlResponse.getResponses()) {
LOGGER.info("XACML Response: " + r.toString());
DecisionType decision = r.getDecision();
LOGGER.info("XACML Decision: " + decision.toString());
LOGGER.info("decision: " + decision.value());
for (Obligation obligation : r.getObligations().getObligations()) {
if (obligation.getObligationId().equalsIgnoreCase("urn:secd:wfs:filter")) {
for (AttributeAssignment aa : obligation.getAttributeAssignments()) {
if (aa.getAttributeId().equalsIgnoreCase("urn:secd:filter:geometry")) {
filterGeometry = aa.getContent().get(0).toString();
}
if (aa.getAttributeId().equalsIgnoreCase("urn:secd:filter:operation")) {
filterOperation = aa.getContent().get(0).toString();
}
}
}
}
}
Sample code for obtaining the response from the PDP
5.1.4.5. Lessons Learned
The Tomcat Filter implementation was based on Java 11. The existing XACML-SDK for Java was available for Java 8. Due to deprecation of javax classes in Java 11, JAXB related functionality had to be updated. The use of GeoTools to create the Filter programmatically could not be achieved. The unresolved problem was that the XML encoder did not include the CRS into the GML part of the spatial filter. Examples and documentation found did only cover non-spatial examples or BBOX Filter. But for implementing the use case, a Disjoint filter with a GML3 geometry had to be constructed. After removing GeoTools completely, a simple string template was used: “<fes:OPERATION><fes:ValueReference>the_geom</fes:ValueReference>GEOMETRY</fes: OPERATION >” where the GEOMETRY was constructed from the response by the PDP (Obligation attributes urn:secd:filter:geometry and urn:secd:filter:operation).
<?xml version='1.0' encoding='UTF-8'?><ns4:Response xmlns:ns6="http://authzforce.github.io/pap-dao-flat-file/xmlns/properties/3.6" xmlns:ns5="http://authzforce.github.io/core/xmlns/pdp/8" xmlns:ns4="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17" xmlns:ns3="http://www.w3.org/2005/Atom" xmlns:ns2="http://authzforce.github.io/rest-api-model/xmlns/authz/5"><ns4:Result><ns4:Decision>Permit</ns4:Decision><ns4:Obligations><ns4:Obligation ObligationId="urn:secd:wfs:filter"><ns4:AttributeAssignment AttributeId="urn:secd:filter:operation" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource" DataType="http://www.w3.org/2001/XMLSchema#string">Disjoint</ns4:AttributeAssignment><ns4:AttributeAssignment AttributeId="urn:secd:filter:geometry" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource" DataType="urn:ogc:def:geoxacml:3.0:data-type:geometry">POLYGON ((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464))</ns4:AttributeAssignment></ns4:Obligation></ns4:Obligations>
Disjoint filter with a geometry constraint
The rewritten Filter was then processed by Geoserver:
typeNames[0] = {http://www.census.gov}poly_landmarks
srsName = urn:ogc:def:crs:EPSG::4326
filter = [[ the_geom within POLYGON ((40.46203574999999 -74.35610985937501, 40.46203574999999 -74.103704953125, 40.674587249999995 -74.103704953125, 40.674587249999995 -74.35610985937501, 40.46203574999999 -74.35610985937501)) ] AND [ the_geom disjoint POLYGON ((40.767774 -73.981464, 40.768268 -73.981396, 40.768483 -73.981634, 40.769272 -73.981131, 40.76983 -73.980652, 40.770488 -73.980215, 40.771096 -73.9798, 40.771753 -73.979298, 40.77241 -73.978862, 40.773018 -73.978447, 40.773708 -73.977923, 40.7743 -73.977465, 40.774859 -73.977072, 40.775565 -73.97657, 40.776288 -73.97609, 40.776947 -73.975588, 40.777554 -73.97513, 40.778244 -73.974671, 40.778852 -73.974234, 40.779427 -73.973776, 40.782105 -73.971899, 40.782795 -73.971375, 40.78342 -73.970917, 40.783995 -73.970437, 40.784685 -73.969956, 40.785293 -73.969498, 40.785983 -73.969018, 40.786624 -73.968537, 40.787264 -73.968036, 40.787922 -73.96762, 40.78848 -73.967119, 40.789105 -73.966704, 40.789713 -73.966245, 40.790353 -73.965787, 40.79106 -73.965241, 40.791684 -73.964804, 40.792358 -73.964281, 40.794247 -73.962905, 40.794905 -73.962403, 40.795545 -73.961966, 40.796153 -73.961529, 40.796761 -73.961049, 40.797418 -73.960612, 40.798125 -73.960109, 40.798782 -73.959607, 40.799374 -73.959149, 40.800047 -73.95869, 40.800425 -73.958428, 40.800507 -73.958124, 40.800588 -73.957885, 40.799509 -73.955312, 40.798298 -73.95248, 40.797003 -73.94954, 40.79669 -73.94952, 40.796329 -73.949761, 40.795705 -73.950241, 40.795031 -73.950744, 40.794374 -73.951159, 40.793684 -73.95177, 40.79306 -73.952142, 40.792419 -73.9526, 40.791729 -73.953103, 40.791154 -73.953496, 40.790414 -73.954042, 40.789199 -73.954937, 40.788624 -73.955352, 40.78795 -73.955854, 40.78726 -73.956335, 40.786669 -73.956815, 40.786028 -73.957339, 40.78542 -73.957732, 40.784796 -73.958212, 40.784188 -73.958627, 40.783514 -73.959086, 40.782873 -73.959588, 40.782233 -73.96009, 40.781625 -73.960548, 40.780852 -73.961029, 40.780294 -73.961466, 40.779587 -73.961946, 40.779012 -73.962383, 40.778388 -73.962863, 40.777747 -73.963343, 40.777106 -73.963845, 40.776334 -73.964391, 40.775726 -73.964871, 40.77502 -73.965438, 40.774494 -73.965939, 40.773771 -73.966398, 40.773196 -73.966856, 40.772523 -73.967315, 40.7718 -73.967817, 40.771225 -73.968253, 40.770585 -73.96869, 40.769992 -73.969148, 40.769368 -73.969607, 40.76871 -73.970065, 40.768135 -73.970501, 40.767511 -73.970981, 40.766837 -73.971397, 40.766213 -73.971898, 40.765605 -73.972313, 40.764981 -73.972793, 40.764389 -73.973251, 40.764621 -73.973791, 40.765651 -73.976428, 40.766812 -73.97926, 40.767575 -73.981008, 40.767774 -73.981464)) ]]Rewritten Filter
5.1.4.6. Follow-Up
The OGC API — Features endpoint was not used as it is not yet possible to pass a Filter parameter. Once the Filter capability is standardized in OGC API — Features — Part 3, the implementation for this demonstration could be adopted accordingly. The implementation would need to be adopted to obtain relevant information such as the feature-type, the request BBOX, etc. according to OGC API — Features.
5.2. Candidate OGC Standards
5.2.1. OGC GeoPose
TBA
5.2.2. OGC API — Records
The OGC API — Records team held two breakouts which focused on advancing the specification for OAB review.
Discussions included:
extension for faceted searching
Record model: contacts and roles
STAC harmonization
5.2.3. OGC GeoParquet
TBA
5.3. Other Specifications
5.3.1. AsyncAPI
TBA
5.3.2. STAC API
TBA
5.4. ASF Apache Projects
5.4.1. Apache Arrow
TBA
5.4.2. Apache Baremaps
TBA
5.4.3. Apache Sedona
Apache Sedona leverages a number of open-source libraries, including GeoTools, to execute some spatial operations. However, as an Apache project, Sedona is striving to reduce its reliance on GeoTools due to licensing issues. In order to address this challenge, Sedona is migrating affected functions to Apache SIS.
In this code sprint, sedona maintainers concentrated on replacing GeoTools’ usage in two key functions within Sedona: ST_Transform and the GeoTiff reader. Their focus was on demonstrating how to migrate these functions to Apache SIS, providing a comprehensive tutorial on the necessary steps involved.
5.4.4. Apache SIS
TBA
5.5. OSGeo Projects
5.5.1. OSGeo GeoNetwork
TBA
5.5.2. OSGeo GeoServer
The GeoServer team worked remote focusing on seperating our implementation into individually downloadable modules. This work is required in anticpation of cql2 being finalized enabling the first public release of a GeoServer ogcapi module.
Pull-reqeust (merged) splitting ogcapi implementation into individual downloads
Pull-request (merged) nightly build docker image allowing public testing of ogcapi modules above
We also checked in with the status of CQL2 which reqmains under active development. We are waiting on this activity before publishing our ogcapi-features module for the general public.
5.5.3. OSGeo OpenLayers
OpenLayers contributors used time at the sprint to discuss topics related to vector feature styling and rendering. They are developing a new WebGL vector renderer and are working on a new internal feature representation and style encoding syntax to work with the new renderer.
OpenLayers currently allows for very flexible feature style generation by allowing users to provide an arbitrary function to generate symbolizers at render time. This limits the efficiency of vector rendering as we are unable to evaluate these style functions outside the main thread. OpenLayers contributors made good progress at the sprint designing a new encoding for rule based styling that allows flexibility in providing expressions to generate symbolizer values while still allowing for efficient evaluation of these expressions in a worker or on the GPU.
OpenLayers contributors will likely plan another dedicated sprint to continue this work and implement the new styling design in the WebGL vector renderer in addition to the existing Canvas (2D) renderer.
5.6. OWSLib
The OWSLib project updated support for OGC API — Features — Part 4: Create, Replace, Update and Delete, as well as upgraded its public documentation to use Read the Docs as well a continuous deploymnent workflow for automated documentation updates.
5.6.1. OSGeo pycsw
The pycsw project upgraded support for STAC API to v1.0.0, and added updates/fixes to its support for OGC API — Features — Part 3: Filtering and Common Query Language (CQL2).
5.6.2. OSGeo pygeoapi
The pygeoapi project completed the implementation of a feature provider for the ERDDAP data server. ERDDAP is a data server that provides a simple, consistent way to download subsets of scientific datasets in common file formats, graphs and maps. This new pygeoaopi feature was created by WMO as part of the https://docs.wis2box.wis.wmo.int project and has been promoted to be included in the pygeoapi core.
Figure 4 — Screenshot of the pygeoapi OGC API - Features provider for the ERDDAP data server.
An update was also made in pygeoapi to highlight licensing when available as part of a collection definition’s link object (where rel=license).
The pygeoapi project implemented the draft OGC API — Pub/Sub conformance class of OGC API — Environmental Data Retrieval standard (see also original discussion paper (23-013)).
Figure 5 — pygeoapi OGC API - Pub/Sub architecture.
Using OWSLib support for OGC API — Features — Part 4: Create, Replace, Update and Delete, the workflow entailed publsihing new features via transactions, which then triggered push notifiations via the MQTT protocol. Pub/Sub capabilities were made available via the AsyncAPI specification, leveraging the OGC API endpoints as subscription channels.
Figure 6 — pygeoapi OGC API - Pub/Sub architecture.
5.6.3. OSGeo QGIS
TBA
5.7. Community Open Source Projects
5.7.1. OSGeo ZOO-Project
The ZOO-Project worked on moving forward OSGeo Incubation as well as the remaining tasks as part of the Project Graduation Checklist to become an OSGeo project.
5.7.2. TEAM Engine
The aim of the code sprint was to migrate TEAM Engine to Java 17 and to get it running with the OGC API Features test suite. This is an important part of the upcoming TEAM Engine 6.0 release which will focus on updating Java to version 17, Tomcat to version 10 and Maven dependencies including Java libraries and Maven plugins.
First, a test setup was built using a Dockerfile which contains Java 17, Tomcat 7, the latest build of TEAM Engine 6.0-SNAPSHOT and the OGC API Features tests suite compiled with Java 8.
Then, the code of TEAM Engine was migrated to Java 17 with the aim that the complete built is successful. This work could be completed and it was possible to build and compile TEAM Engine with command mvn clean install site -DskipTests using Java 17. All results were documented in issue 511.
Afterward, TEAM Engine compiled with Java 17 was tested with OGC API Features tests suite using the above described test setup. No errors were detected during testing. The TEAM Engine webapp started, a login via Web Browser Interface was possible and a test run with OGC API Features test suite could be executed. The generated report was displayed as expected and test details could be viewed. Thus, the aim of the code sprint was reached.
In the remaining time, the work concentrated on further tasks being important for the TEAM Engine 6.0 release.
Identified open tasks were documented in separate issues:
Unit test ImageParserTest.parsePNG_noAlphaChannel fails with Java 17
Resolve errors and warnings of maven-javadoc-plugin when using Java 17
Analyse warnings logged by maven-pdf-plugin when using Java 17
Finally, all of them were summarized in milestone 6.0 of TEAM Engine.
5.7.3. OSGeo Leaflet
TBA
5.7.4. Camptocamp ogc-client
TBA
5.7.5. HomeAssistant-SensorThings
The SensorThings standard proved approachable enough so that HomeAssistant-SensorThings could be developed within the span of the codesprint, albeit not being feature complete.
Figure 7 — Partial screenshot of a Home Assistant dashboard showing SensorThings observations from the British Geological Survey, from their FROST server.
The SensorThings standard has some traits that proved very helpful for the development and debugging of the client: the usage of JSON over HTTP, and the availability of full REST-like URIs within the metadata offered by the endpoint.
The nomenclature of concepts posed a minor challenge. An OGC Datastream is a HA sensor, and a OGC Thing is a HA device; and an OGC API endpoint becomes a config entry in HA. The concept of an OGC Observation is implicit in the behaviour of a HA sensor. The OGC concepts of Sensor, Location, FeatureOfInterest and ObservedProperty do not have an equivalent on HA. The HA concepts of area, zone, scene and others do not have an OGC equivalent.
Several concerns have been raised:
Units of measurement are inconsistent across endpoints. A temperature sensor from USGS uses degC, one from Fraunhofer uses °C, and one from BGS uses C. A human will interpret all those as “degrees celsius”, but this mismatch makes automated data comparisons impossible. The standard recommends using UCUM (Unified Codes for Units of Measure), but none of the SensorThings services explored were following this recommendation.
The values for the description and definition of ObservedProperty are unreliable, even though they are mandatory fields. All SensorThings services explored were either using empty strings, or reusing the name of an ObservedProperty as its description and/or definition. No servers produce URIs for the ObservedProperty definitions, as mandated by the standard.
Home Assistant focuses on realtime or quasi-realtime data, and the only viable strategy for SensorThings client is to periodically poll for new observations. However, there is no estimation on how often new observations are made, not even a rough one. Some sensors can provide new observations every second, while others might provide observations once every hour, or once per day. Polling data each second from a daily sensor is a waste of networking resources, whereas polling each day from a secondly sensor introduces an obvious data lag. It’s impossible to choose the right approximate rate of polling. The MQTT extension of the SensorThings specification alleviates this issue, but complicates the implementation of both servers and clients.
The Home Assistant architecture relies on communicating the last known state of a sensor/device/service; this is arguably the most common use case for internet-connected sensors. However, SensorThings provides no specific facility for this. Instead, the generic way of fetching any and all historical observations must be used, including a query string like ?$top=1&$orderby=phenomenonType desc. The specification states that «the SensorThings API is designed specifically for the resource-constrained IoT devices and the web developer community», but it can be argued that query languages are ill-suited to be implemented in constrained device.
SensorThings is different than other contemporary OGC APIs (such as Maps, Features or Coverages) in that it doesn’t implement content negotiation. It would be interesting to consider content negotiation as an extension, so the last observation could be fetched in human-readable HTML form, or so a series of ovservations could be downloaded as a machine-readable CSV.
Finally, it would be possible to implement a SensorThings server on top of a Home Assistant installation; the scope of such development falls out of a 3-day codesprint, but could be achieved in a timeframe of several weeks. This would provide a cheap way to provide SensorThings endpoints for domestic-grade sensor platforms such as weather stations or soil moisture sensors.
5.7.6. Maplibre
TBA
5.7.7. MDME
TBA
5.7.8. OL-Cesium
During the code sprint, the participants fixed support for the latest Cesium version 1.104. Several of the examples were fixed. Another fix updated the code base so that the Parcel bundler could support better reading of the code base. Support for ECMAScript 6 (ES6) exports was also fixed. The participants also worked on the descruction method to improve memory handling. The participants also worked on the migration to Typescript. Another archievement was the work done on adding polygon extrusion support. Another feature that was implemented during the code sprint was OL-Ceisum support in Geomapfish.
5.7.9. OSGeo pygeometa
The pygeometa project added support for detailed schema capability reporting, so that a client can be aware of which schemas are supported, and whether those schemas have read and/or write support. This was also implemented as pygeoapi plugin, and made available as part of the pygeoapi public demo. “pygeometa as a service” is used by the live demo of the mdme project. The schema capability integration was added by mdme as part of the sprint.
The MDME project uses vjsf internally to render a web based metadata edit form derived from json schema. A visual edit form for MCF has proven to be usefull in yaml oriented metadata workflows for some user groups. Some json schema concepts such as any-of are not supported by vjsf. During the sprint MDME has been improved to prevent the use of any-of and selector patterns having ‘*’. As mentioned above, import from iso19139 and export to iso19139, stac, and ogcapi:records (based on a transformation service provided by pygeoapi/pygeometa) have been introduced.
5.7.10. Geomapfish
TBA
5.7.11. Smapshot
A screenshot of the application, shown during the code sprint, is presented in Figure 8. As illustrated Smapshot enables users to geolocalise historical images in 3D.
Figure 8 — A screenshot of Smapshot
During the code sprint, Geopose format was tested as a way to return position and orientation data of the 3D geolocalized images from smapshot from a standard OGC API features. The Geopose data format has been added to the Open Source NodeJs server of OGC API features: CodeSprint PR. A quick data visualisation has been implemented in Cesium to display the geopose of the camera retrieved from the API with smapshot data and the geopose of the photograph. A screenshot of the application is shown in Figure 9.
Figure 9 — Position and orientation from a geopose represented in Smapshot
5.7.12. go-ogc
Work was started to add OGC API — Records support to the library. This involved adding an extension mechanism to the existing OGC API — Features structs, in the same way that since the Records spec extends the Features spec. The library is being used in an internal service at Planet Labs, where the OGC API — Features spec is being used to provide metadata about Planet’s imagery catalog. The records branch of the go-ogc library is being used to test an update to the internal service that adds support for OGC API — Records.
5.7.12.1. Suggestions
5.7.12.1.1. Add conformsTo property to OGC API — Features
The OGC API — Records specification extends OGC API — Features by adding a conformsTo property to GeoJSON records (or features). This would be a useful extension point for other specifications in addition to Records (e.g. STAC API or other standards that might extend Features). Instead of being defined in the Records spec, it would be useful if the Features spec defined the conformsTo property as a common way to advertise conformance with other specifications that might extend Features.
5.7.12.1.2. Weigh the pros/cons of adding top-level GeoJSON Feature members
The OGC API — Records specification adds top-level members to GeoJSON Features. For example, the time member is added at the top level of the Feature object. By contrast, members like created and updated are added to the properties object. The location of these members will determine how they can be accessed by existing software that works with GeoJSON. It would be a good idea to evaluate the pros and cons of adding top-level members to GeoJSON Features and to consider whether it would be better to add these members to the properties object instead.
For example, using Fiona (which delegates to OGR/GDAL for feature access), the time member cannot be accessed when reading a feature that looks like this:
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [0, 0]
},
"properties": {
"name": "Null Island"
},
"time": "2023-05-10T13:00:00Z"
}
GeoJSON Feature with top-level time member
By contrast, the time member can be accessed when reading a feature that looks like this:
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [0, 0]
},
"properties": {
"name": "Null Island",
"time": "2023-05-10T13:00:00Z"
}
}
GeoJSON Feature with time in the properties object
The same limitation applies to mapping clients that might allow filtering or styling features based on feature properties. In these cases, the top-level members are not useful. While updates to these existing libraries can be made to handle this new extended type of GeoJSON, it is likely to slow adoption, and the spec may find more interoperability if it were designed to work with existing libraries.
5.7.13. GPQ
The GQP source was recently open sourced, and is now available under the Planet Labs GitHub organization: https://github.com/planetlabs/gpq. A CI workflow was added to publish a WASM binary of the utility to GitHub Pages, allowing converstion of GeoJSON to GeoParquet and vice versa in the browser. In addition, work was done to improve how the Parquet schema is derived from GeoJSON features. Previously, the schema would be derived from the first feature in a collection with all non-null property values. Now, the schema is incrementally built over a range of features, allowing for a more complete schema to be derived. This is especially useful for large collections with sparse properties.
6. Discussion
6.1. GIGS
The GIGS tests can be found on the gigs.iogp.org website. The participants from Geomatys worked on PROJ-JNI to enable it to execute additional GIGS tests. PROJ is a coordinate transformation software package, built using C/C++ library, that transforms geospatial coordinates from one CRS to another. PROJ-JNI is a Java Native Interface for PROJ.
Figure 10 — Example execution of the GIGS test sofware
Not every software product will support every feature that the GIGS tests specify, so a properties file is provided to enable the user to enable or disable specific aspects of the tests. PROJ-JNI is an implementation of the OGC GeoAPI Standard.
6.2. Maplibre
The rendering pipleline is generally not exposed to the user. So the participants working on Maplibre experimented with accessing the rendering pipeline. They implemented functionality to export the depth buffer (shown in the top left of Figure 11). The approach offers the ability to create light effects and special environmental effects such as an atmospheric haze (shown in the bottom right of Figure 11). So the participants configured their prototype so that when the viewing point is far away the effect is disabled. The participants also examined other parts of the pipeline so that any object that is rendered is based on the modified depth buffer. This included for example modification of the coordinate buffer (shown in the bottom left of Figure 11) so that tiles at the top of the screen represent a large space. The third aspect that can be modified using this approach is the color of the image, for example to enable a user to adjust the brightness or to create aesthetic visuals (shown in the top right of Figure 11).